A critical review of executive coaching research: A decade of progress and what’s to come

This paper aims to summarise the current state of coaching research and to provide a basis for future research which will provide a frame of reference ensuring that research builds on previous studies and adds to knowledge rather than replicating previous findings in innocence. This approach will prevent wasted effort and resources in organisations and research. The paper is divided into three sections. The first two sections review the state of research over the past hundred years, with a greater focus on the past decade when coaching research has accelerated at warp speed. The paper divides the recent research into categories; the nature of coaching, coach behaviour studies, client behaviour studies, relationship studies and executive coaching impact studies. The third section considers the future direction research may take. It identifies key questions which the authors believe should be the focus of future research and highlights the work undertaken to support coaching researchers and published by the Coaching Foundation. Introduction It has been ten years since Kampa-Kokesch & Anderson’s, (2001) seminal review of coaching research. This article highlighted the scarcity of coaching research and noted seven impact studies of coaching research had been published in the psychological literature. More generally coaching research can be traced back to 1937 when the first impact study of coaching’s impact on manufacturing was published by Gorby (1937). The study, while limited in its methods, was a marker signalling the potential of coaching as a force for good within organisations. However, after a short paper building on Gorby’s study (Bigelow, 1938), the trail went cold during the 1940’s and did not really start again until the 1990’s and the work of writers such as Kilburg, Diedrich, Lowman and others in Consulting Psychology, which has blazed the trail in this area of psychology practice. A closer look at many of the early studies noted by Kampa-Kokesch & Anderson revealed weaknesses with methodology. These weaknesses included limitations in the research methods, inadequate sample sizes and studies where the claims made were not fully supported by the data. Such research problems however are not uncommon for new domains, of which coaching in 2001 was certainly one (Passmore & Gibbes, 2007). If we consider the evolution of other disciplines we can draw lessons from how knowledge evolves. As a new area evolves it passes through several phases prior to maturation. Initially it tends to focus on defining what it is concerned with. This involves a process of exploration and sharing of the phenomena and what is experienced by practitioners. This first exploration phase helps to shape and identify what is understood of the field from experience and what can be considered within the field of enquiry and what cannot. After the exploration phase, attention shifts to theory, methods and measures. Researchers seek to develop and test new interventions, products or protocols. The initial part of this phase is often marked with case studies and small qualitative research. This gradually shifts towards theory building and random control trail studies with large sample sizes, and finally to meta analysis. A third phase is characterised by concern with exceptions and variance to the theories. One theme within this is the question: Which groups or issues benefit most from which approach? Each phase requires different methodologies and instruments. As a result we would expect to see a maturing of the research undertaken. A shift from exploration through survey and case studies to theory development using grounded theory and similar qualitative techniques to quantitative studies using random control trails and ultimately to meta-analysis studies. This perspective allows a framing of the literature from a more appreciative stance and allows us to consider emerging work and research in terms of its contribution to the current stage in maturity. We would like to consider coaching research in light of this maturity model and use it to help frame an appropriate and stimulating journey forward. In this article we aim to review the state of coaching research and ask “Where next for the research agenda”? In this sense our ambition is to provide a foundation and maybe a compass for coaching researchers. Purpose of coaching research As researchers we have both been challenged in the past by practitioners; ‘So why is research important, I know it works and that is enough’. For many, that is enough. However when decisions need to be made about the return on investment (ROI) of consulting, coaching and training, then both HR and psychologists need to ensure that they can demonstrate the contribution of coaching to the organisation’s bottom line. Like many other HR interventions, tangible costs are not the whole story, or at least we would argue they should not be. A price cannot be placed on the saving of a life from a road traffic death or the improvements in hope, resilience, emotional intelligence (Passmore & Brown, 2009). So in short coaching research provides some of the evidence of coaching’s value in cash and others in non-cash terms for individuals and their organisations. We would also argue that this can provide valuable benefits for us as coaches. Specifically one of the outputs of our research is to identify and define the knowledge base upon which we work – what is our unique combination of knowledge and skills which delineates what we do from other helping and learning interventions? This is of particular importance at a time when some countries are legislating for competencies expected of those working within the area. It is also essential for any consideration of coaching as a profession – the very definition of a profession includes having a defined and unique knowledge base. With this idea of an evolving and developing knowledge base supplied by appropriate and stimulating research there comes the concept of sustainability of practice. The bench mark for good practice will continually develop at a pace in line with the rapidly changing environment of our clients. Coaches need to be able to interrogate that practice to identify what is fit for purpose within their own work. Each coach becomes their own researcher into their own practice and the developing literature. In line with the experience of other professions the need for professionalism to include a requirement of the practitioner to be researcher is clearly appropriate. Thus training and development becomes life long, building upon a sound technical base. With increased demand for coaching, new coaches need to be trained, and with this, decisions need to be made about what to train (Spence, Cavanagh, & Grant, 2006). Research should help us identify the skills which make a difference in the coaching relationship. Do questions make a difference? If so how? Or should the coach just tell the client what to do. Does it matter if the coach moves rapidly from one to the other, or is consistency in style important? Lastly, and maybe most importantly, working with others in intimate relationships, such as coaching, places an ethical obligation on the coach. The coach needs to ensure that their interventions do not expose the client or the coach to harm. The training of coaches needs to include consideration of ethical issues as well as the importance of supervision or other reflective practice methods. The coach needs to be sufficiently trained to identify the boundaries of their competence and manage these within the contract. This should include identification of medical disorders from depression to narcissism or anti social personality disorder, where coaching is not an appropriate or helpful intervention. In the next sections we review of the early research which took place during the 20th century and move onto consider the research published after Kampa-Kokesch & Anderson (2001) during the past decade. Review of Research: 20th century The period from 1937, the date of the first identified coaching study, to 1999, was a period of slow progress, which saw more published papers in the final ten years than in the preceding fifty. Much of this is due to the leading work of this journal and of Division 13 of the APA which identified the growing trend of its members working in organisations. More recently the journal has been joined by the British Psychological Society Journal, the International Coaching Psychology Review and by commercial journals such as Coaching: An International Journal of theory, research and practice. In the 62 years between 1937 and 1999 there were a total of 93 articles, PhDs and empirical studies published. The 1937 and 1938 papers were followed by a slow trickle of papers. One research paper was published in the 1940’s (Lewis, 1947) and this was followed by nine studies in the 1950’s, the majority concentred in the later half of the decade. This was followed by three studies in the 1960’s and three in the 1970’s. It was not until the 1980’s that the first signs of growth were seen. Several of these early papers hinted at the potential that coaching may have either as a separate organisational intervention, or as a complimentary intervention to help in skills transfer after training, for example Holoviak, (1982) study which examined training programs in relationship to variations in company productivity levels in the coal industry. The studied used a semi structured interview method and identified that companies which provided greater amounts of management and supervisory training including coaching also achieved higher productivity. It was not until the 1990’s that coaching research papers became a common occurrence in the literature with 41 papers, PhD’s cited by PsycINFO and Dissertation Abstracts International for the period. The focus of the papers starts to widen, with a recognition of the role of coaching in enhancing feedback (Hillman, Schwandt, & Bartz, 1990), the contribution that coaching can make to both leadership (Popper & Lipshitz 1992) and management (Graham, Wedman & GarvinKester, 1993 & 1994). One of the most interesting and rigorous studies during this period was a triangulation and psychometric based study of coaching efficacy (Peterson, 1993b). The PhD research found that participants improve by about .85 standard deviations in overall effectiveness as a result of their coaching programs. The overall focus however was coaching as a management skill and with a case study led methodology as opposed to quantitative methods comparing different interventions and using control groups. Where qualitative methods were used the favoured methodology was content analysis compared with the more sophisticated methods of Interpretative Phenomenological Analysis (IPA), Grounded Theory or Discourse Analysis. However, there was wide inconsistency in the quality of case study papers, and one of us has suggested ways this could be improved (Passmore & Gibbes, 2007). This feeling our way into enquiring about coaching can be considered appropriate for the exploration stage of the literature’s development. Sharing of case studies with their qualitative richness has allowed the identification, and to some extent validation, of what coaching looked like within these contexts. This was about to change in the coming decade as the research questions changed and theory was developed; coaching research started to apply the methods and rigour which counselling, training and other organisational interventions had been during the 1980’s and 1990’s. The work of Olivero, Bane and Kopelman, (1996) was the first reported attempt at examining the influence of coaching in a public sector municipal agency. Thirty one managers underwent a conventional managerial training programme followed by 8 weeks of oneto-one coaching by internal coaches. Action research was employed to examine the influence of coaching upon translating the training into behaviour change within the workplace. The study used measures of productivity appropriate to a work based project chosen as the focus of the coaching and found a 22.4% increase in productivity after training and 88 % increase after coaching. Looking back on this study reveals considerable weaknesses in the methodology, but the paper was an important milestone. Review of Research: 2000 –2009 The nature of coaching By 2000 the initial exploration of the field had provided various definitions of coaching and attempted the delineation of what constituted coaching within the leadership development portfolio (Judge & Cowell 1997; Thach & Heinselman, 1999). It was suggested that executive coaching was really a repackaging of activities and techniques borrowed from other disciplines such as counselling, psychology , learning and consulting (Tobias, 1996). It is undoubtedly true that a significant amount of the knowledge base used by coaches originates from other disciplines. However, in our view, this does not negate the uniqueness of the synthesis of these elements to produce an offer of benefit to clients and one which is not provided by other interventions. Indeed we hold the view that psychological knowledge and practice is an integral part of coaching as it is of all relationships. By this we simply mean that the coach needs to consider the behaviour, cognition and emotion of the client, and use this information to help in the process of learning and change. Several papers have reviewed and debated the nature of coaching and its boundaries with counselling (Bachkirova & Cox, 2004; Passmore, 2007a), as well as the emerging domain of coaching psychology (Stewart, O’Riordan & Palmer, 2008; (Sperry, 2008). That being said there is as yet no agreed definition of coaching and the research focus has moved on. Examples include: “a collaborative and egalitarian relationship between a coach, who is not necessarily a domain-specific specialist, and Client, which involves a systematic process that focuses on collaborative goal setting to construct solutions and employ goal attainment process with the aim of fostering the on-going self-directed learning and personal growth of the Client” (Grant & Stober, 2006) To the more organisational perspective offered by Kilburg: “a helping relationship formed between a client who has managerial authority and responsibility in an organization and a consultant who uses a wide variety of behavioural techniques and methods to help the client achieve a mutually identified set of goals to improve his or her professional performance and personal satisfaction and, consequently, to improve the effectiveness of the client’s organization within a formally defined coaching agreement” (Kilburg, 2000, p 142). In reflecting on the research and publications over the past decade we would offer the following broad definition of coaching: “ a Socratic based dialogue between a facilitator (coach) and a participant (Client) where the majority of interventions used by the facilitator are open questions which are aimed at stimulating the self awareness and personal responsibility of the participant”. As with all definitions there is the potential for debate. One challenge for such a definition is the lack of recognition around group and team coaching. We hold the view that one to one coaching is a different activity from group and team coaching, due to the intimacy and candour that can be created in the one to one relationship. This is not to say that coaching techniques cannot be used with groups, but rather that the dynamic which is created is different. The research on team coaching is at a lower level of maturity but there is a developing literature within the realm of team effectiveness (Mathieu, Maynard, Rapp, & Gilson, 2008). Specifically (Wageman, 1997, 2001) has made a substantial contribution culminating in the publication of a theory of team coaching with Hackman (Hackman & Wageman, 2005) The model focuses on the functions that coaching serves for a team, rather than on either specific leader behaviours or leadership styles, identifies the specific times in the task performance process when coaching interventions are most likely to have their intended effects, and explicates the conditions under which team-focused coaching is and is not likely to facilitate performance. It may be argued, that group or team coaching is too close to Action Learning Sets and group facilitation to usefully distinguish between them. Indeed the use of such methodology has been activity explored and described in the team coaching context (Vaartjes, 2005) and research studies such as these identified here may illuminate that question. For the present the question is still unresolved and hence we do not address this mode of coaching explicitly in this paper. Whilst considering what coaching is the community has also explored the concept of process within the coaching engagement i.e. what would be seen to be happening. At first the studies were relatively naïve and tended to err towards marketing literature on a particular model or tool. A more critical engagement with the process was needed and one bright light here was a PhD study by Dingman (Dingman, 2004) where the literature review compared a series of different coaching processes and identified six generic stages which were part of all published models: 1. Formal Contracting 2. Relationship building 3. Assessment 4. Getting feedback and reflecting 5. Goal setting 6. Implementation and Evaluation The relative weighting of each of these stages and their exact titles may change but in all the models reviewed each stage was present. The perspective which underpins the process i.e. the five major approaches to executive coaching interventions that have been summarised by Peltier (2001): as psychodynamic, behaviorist, person-centered, cognitive therapeutic, and system-oriented and explored within (Feldman & Lankau, 2005) recent review. Each have been further explored within coaching (Cocivera & Cronshaw, 2004; Ducharme, 2004; Hrop, 2004; Kilburg, 2004; Sherin & Caiger, 2004). As befits the explorative nature of the enquiry these studies have tended to use case study and surveys as the methodology and instruments of choice as we explore what is actually happening out there and what people are doing. In the following sections we look at the work from the viewpoint of each actor within the intervention. Coach behaviour studies Numerous authors have tried to identify the critical attributes of the effective coach (Kilburg, 1996, 2001). One of us (Jarvis, Lane, & Fillery-Travis, 2006) identified three areas as being critical – self awareness, core coaching competences and an understanding of the ethics and management of coaching relationships. Other studies (Dingman, 2004) have shown similar competencies but perhaps with more clear delineation i.e. interpersonal skills, communication skills and instrumental support which include effects such as creativity, dealing with paradox etc. There is a continual debate here as to the academic requirements of coach training and its content. Specifically should coaches hold a degree in psychology? Opinions vary from absolutely! (Berglas, 2002) to the realisation that our clients probably want a mixture of all; graduate training in psychology; experience in, or understanding of, business; established reputation as a coach; listening skills; and professionalism as expressed by intelligence, integrity, confidentiality and objectivity (Wasylyshyn, 2003). It is clear that there is a role for the professional bodies within this arena specifically as they approach individual accreditation of professional coaches. Their construct of what differentiates the seasoned practitioner from the novice other than simply hours of practice will continue to influence both the training received by coaches and the continuing professional development they will choose. Client behaviour studies It is perhaps not surprising that there have been a range of studies looking into how the client’s behaviour impacts upon the effectiveness of coaching. It is clear that a willing and informed client will get more from the encounter when coaching is seen as important, relevant and beneficial. This has been explored within the CIPD research (Caley et al, 2002) where the motivation to learn was identified as one of the most critical factors influencing learning effectiveness. Readiness for change is therefore a prime factor in predicting outcomes. Several authors have tried to extend this analysis to consider if any specific sector of society delineated by gender (Vinnicombe and Singh, 2005), learning style or personality type (Dawdy 2004) benefits more from coaching then another. However to date none have been able to provide evidence of any enhanced efficacy. CoachClient Relationship studies It is now recognised that the most consistently identified factor seen as contributing to the success of a coaching engagement is the quality of the relationship between the coach and client. (De Haan, 2008a & 2008b). This is in agreement with studies from related fields such as psychotherapy where the ‘Common factors such as empathy, warmth, and the therapeutic relationship have been shown to correlate more highly with client outcome than specialized treatment interventions.’ (Lambert & Barley, 2002). Initial investigations of coaching interventions started with Wasylyshyn who undertook a survey of clients and found the highest-scoring characteristic of an effective coach was the ability to form a strong connection with the client. It should be noted that this study was carried out on the clients of this one coach and therefore cannot be viewed as definitive (Wasylyshyn, 2003). However this issue has since been the subject of a variety of studies (Thach, 2002; Dingman, 2004) most of which are looking at efficacy more generally but the most recent empirical study was undertaken to investigate the links between the coach-client relationship and the success of the intervention. 73 managers and 24 coaches were involved in the work and 31 coach-client dyads were analysed. The results indicated that the relationship pays a role between the coaching received and the development of self-efficacy (Baron & Morin, 2009). Coaching impact studies: Organisational As the process and perspective underpinning coaching has become clearer the focus of research has shifted to the second phase of theory development by looking at the factors which contribute to effective coaching and the overall impact upon the individual and organisations. We have previously categorised these factors in terms of coach attributes, client attributes, the organisational context for coaching and coaching process (Jarvis et al., 2006). The differentiation of factors and their weighting requires more sophisticated methodologies and with this, the research resources to make it happen. Thus research involving controlled trials is moving to the research organisations such as the Universities and large scale consultancies. The increase in doctorate level research in coaching provides the resources and timescale appropriate to these more ambitious studies. One such study used a quasi-experimental pre-post control group design to examine the impact of coaching on individual leadership development beyond what might be expected from attending a leadership development program only (HernezBroome, 2004) . It was found that even a single phone conversation a month for three months with an experienced coach provided significant benefits in producing behaviour change within the workplace. Most studies discussed above have tried to identify the impact within a single study. The impact has varied from study to study, but a recent paper (De Meuse & Dai, 2009) has undertaken the first meta-analysis study. The paper drew on a very limited range of studies, six in total. (Evers, Brouwers, & Tomic, 2006; Luthans & Peterson , 2003; Peterson, 1993b; Smither et al. 2003; Togel and Nicholson , 2005; and Wolfred, 2003 ). It concludes that previous claims of ROI were over stated, but that coaching does yield a relative good ROI based on the six studies, four of which were used (Table 1). Table 1: Statistics Reported in the Coaching Studies |Study |Self-Ratings Effect Size (corrected for unreliability) | | |Skill/Performance | | | | |Improvement | | | |Peterson (1993) |1.98 (N = 100) | | | |Luthans & Peterson (2003) |0.02 (N = 20) | | | |Evers et al. (2006) |0.34 (N = 30) | | | |Wolfred (2003) |0.46 (N = 23) | | | | | | | |Others’ Ratings Effect Size (corrected for unreliability) | |Study | | | |Skill/Performance | | | | |Improvement | | | |Peterson (1993) |1.83 (N = 100) | | | |Luthans & Peterson (2003) |1.41 (N = 100) | | | |Smither et al. (2003) |0.06 (N = 382) | | | |Togel & Nicholson (2005) |0.65 (N = 89) | | | (De Meuse & Dai, 2009) The ROI however varies between the estimates of clients and those of their managers. The true effect size on ROI corrected for sampling error in the four studies was 1.27 compared with 0.6 for the effect size in others ratings. However a closer examination of the data in Table 1 shows wide variation between 1.98 and 0.02 for self rated improvement and for others ratings from 1.83 to 0.06. These variation as wide and not consistent and thus questions need to be asked whether conclusions can be reached from the results. The authors warn however that coaching effect may be situational, with stronger effects demonstrated with specific individuals and domains. This would accord with Lambert and Barley (2002) who from a meta-analysis of counselling noted that client readiness the most significant factor in bringing about change, accounting for 40% of the variation in outcomes. As yet we cannot say conclusively what factors may lead to more positive outcomes. The Future decade for coaching research Research Methodology We can see that as we emerge from the exploration and definition phase within the research field we are at the point where theory development and testing comes to the fore. With it we are seeing a shift from case study and uncontrolled trials to designs appropriate to the type of research questions prompted by theory generation (what coaching for what client?) By 2021 we hope that researchers across the globe will have completed fifty to hundred large sample size studies (with sample sizes over 60 participants in each group) using 2 or more conditions including a control, random allocation of participants to different conditions and placebo interventions alongside the control no intervention and the coaching intervention. As identified previously, these studies will be undertaken at the doctoral level, allowing greater sophistication and longer time frames for pre, post and 6 or 12 months post intervention measures. These methodologies will allow coaching to reveal its impact over the course of the intervention and beyond. Initial studies suggest that programme lengths of 4 to 6 months are impactful but programmes of over 1 year yield diminishing returns (Luthans & Peterson, 2003). But we must be sophisticated enough in our thinking to appreciate that just as case study as a methodology can only provide a limited perspective in our exploration of coaching so random controlled trials are not appropriate for issues requiring depth and theory generation. As we go forward the richness of our questions will develop and in addressing them we will need to bring to bear the full armoury of research paradigms, approaches and methodologies. We believe that all research paradigms have a place within coaching and that the only criteria should be one of research excellence the congruence of paradigm, question, approach, methodology, instrument, analysis and conclusion. Qualitative studies using recognised techniques such as IPA, Grounded Theory and Discourse Analysis have a valuable role to play in helping us understanding the human interactions of coaching at a deeper level. We also would argue the case in favour of mixed methods studies which call upon both traditions and through triangulation between qualitative, quantitative and existing research literature to offer new understandings. This inclusive stance must also extend to who contributes to the research. We strongly urge our community to not let the academic – practitioner divide appear within coaching. This separation of research from practice has been the cause of lost opportunity and internal discord within professions as widely dispersed as teachers, marketing professionals and IT workers. The major concern for all of us is that such divides have resulted in a substantial reduction in research sponsorship from major stakeholders on the grounds that research is not relevant to their practitioners and the results are not valid within their organisations. An example of how academic/practitioner collaboration will add value and validity to even large scale research is the measure of ROI. In institutions, the only measure that matters is ROI (Phillips, 2005). Return on Investment however is notoriously difficult to measure in human settings, where factors are difficult to isolate. As a result the studies appear to exaggerate the ROI (De Meuse and Dai, 2009). With further and better studies a more realistic ROI can be generated using sixty studies rather than six as has been undertaken in counselling and other domains. Even this meta ROI however will hid variation, and with greater studies we may be able to get behind the factors which influence the success of coaching projects, such as top team commitment. We may also get a better understanding of ROI depending on grade, so while Csuite coaching may yield 0.7(based on external assessment), middle manager coaching may yield 0.3, and lower grade coaching may only be cost effective when undertaken internally by internal coaches. The experience of practitioners within such programme will vastly enhance the relevance and impact of the resulting studies.